1. Field of the Invention
One disclosed aspect of the embodiments relates to an image processing apparatus, an image processing method, and a storage medium.
2. Description of the Related Art
RAW and Joint Photographic Experts Group (JPEG) are well-known file formats generated by a digital camera.
A sensor of the digital camera often employs a Bayer array including a two-dimensional array of R (red), G (green), and B (blue). In image data in a RAW format, each pixel includes a signal of a color corresponding to any of color filters according to the two-dimensional array of the color filters of an image sensor. A file in the RAW format is suitable for a case in which white balance processing for adjusting a balance of signal intensity among colors corresponding to the color filters or lateral chromatic aberration correction for adjusting an aberration due to a deviation of image forming positions among the colors are performed. For example, Japanese Patent Application Laid Open No. 2008-015946 discusses the lateral chromatic aberration correction is performed after image data in the RAW format is separated into image data made of signals of respective colors.
The image data in the RAW format has large data capacity and cannot be opened with a general-purpose image display viewer. Therefore, typically, interpolation processing for causing each pixel of the image data to have signals of all of the R, G, and B colors is performed, and a luminance signal (Y) and color-difference signals (U, V) are generated from the image data after the interpolation to generate a file in a JPEG format made of Y, U, and V image data.
Then, in the process for generating image data in the JPEG format from the image data in the RAW format, a high-frequency component is extracted from the luminance signal Y and edge enhancement processing is applied thereto, or an interpolation direction is determined when the color-difference signals U and V are generated so that false color suppression processing is applied.
In recent years, there is a demand for changing parameters of the white balance processing or the lateral chromatic aberration correction with respect to the Y, U, and V image data.
Therefore, in such a case, it can be considered to convert the Y, U, and V image data into R, G, and B image data, and to execute the white balance processing or the lateral chromatic aberration correction again with desired parameters.
However, a digital camera is often configured from the above-described image processing circuit using dedicated hardware for acceleration of the processing. That is, a camera including an image sensor of the Bayer array is provided with a module that expects an input of image data in the RAW format obtained by the image sensor of the Bayer array.
For example, when image data in the JPEG format is configured in a YUV422 that is a YUV format, one luminance signal Y is allocated to each pixel and one color-difference signal U and one color-difference signal V are allocated to each of two pixels adjacent to the each pixel, in the image data. Therefore, the configuration is different from that of the Bayer array in which one of R, G, and B signals is allocated to one pixel, and the module configured for the Bayer array cannot process the image data of the YUV422 in the JPEG format. A similar problem is caused not only in the digital camera, but also in a device provided with a module configured for the Bayer array. In addition, for a similar reason, a problem that the Y, U, and V image data in the JPEG format cannot be processed is caused not only in hardware, but also in an image processing apparatus that only includes software incorporated for the Bayer array.
Here, it can be considered to convert the image data of the YUV422 into image data in the RAW format of the Bayer array by generating the R, G, and B signals in each pixel from the image data of the YUV422, and thinning out the image data for each color. A method for generating the image data in the RAW format of the Bayer array from the image data of the YUV422 will be described with reference to FIGS. 8 and 9.
FIG. 8A illustrates an example of a configuration of the image data of the YUV422. One luminance signal Y is allocated to each pixel, but one color-difference signal U and one color-difference signal V are allocated only to each of the two pixels adjacent to the each pixel. Therefore, the color-difference signals U and V are interpolated in the horizontal direction with respect to the image data illustrated in FIG. 8A, so that image data in which the Y, U, and V signals are allocated to each of the pixels is generated as illustrated in FIG. 8B.
R, G, and B signals in each pixel are calculated from the Y, U, and V image data after the interpolation illustrated in FIG. 8B according to the following formula 1:R=1.000Y+1.402V G =1.000Y−0.344U−0.714V B=1.000Y+1.772U   (formula 1)
FIG. 9A illustrates image data where the R, G, and B signals in each pixel have been calculated. A signal of a color corresponding to a color filter at a position of each pixel is selected from the image data illustrated in FIG. 9A to accord with the Bayer array. In doing so, image data of the Bayer array illustrated in FIG. 9B can be generated. Then, the image data of the Bayer array illustrated in FIG. 9B is again input to the module configured for the Bayer array, so that the white balance processing or the lateral chromatic aberration correction can be executed with different parameters.
Image data to which the white balance processing or the edge enhancement processing have been applied with the new parameters is again converted into image data in the JPEG format made of Y, U, and V. In generating the Y, U, and V image data from the R, G, and B image data of the Bayer array, it is necessary to perform calculation using the R, G, and B signals for the same pixel. Therefore, to generate the Y, U and V image data from the image data of the Bayer array illustrated in FIG. 9B, the interpolation processing for causing each pixel to have signals of all of the R, G, and B colors is performed as illustrated in FIG. 10.
The signals indicated by R′, G′, and B′ in the image data illustrated in FIG. 10 are signals obtained by the interpolation processing. The signals obtained by the interpolation processing are values obtained by weighted addition of neighbor signals of the same color and are values presumed from the neighbor signals. Accordingly, the signals have poor reproducibility in minute signal level change. Therefore, a band of the image data illustrated in FIG. 10 after the interpolation processing for R, G, and B is lower than that of the image data illustrated in FIG. 9A before the interpolation processing for R, G, and B, although both of the image data include all of the R, G, and B color signals in each pixel in a similar manner. That is, if the Y, U, and V image data is simply converted into the R, G, and B image data of the Bayer array and the R, G, and B image data is returned to the Y, U, and V image data again, the band of the image data is decreased.